Deep Haptic Model Predictive Control for Robot-Assisted Dressing

Robot-assisted dressing offers an opportunity to benefit the lives of many people with disabilities, such as some older adults. However, robots currently lack common sense about the physical implications of their actions on people. The physical implications of dressing are complicated by non-rigid garments, which can result in a robot indirectly applying high forces to a person's body. We present a deep recurrent model that, when given a proposed action by the robot, predicts the forces a garment will apply to a person's body. We also show that a robot can provide better dressing assistance by using this model with model predictive control. The predictions made by our model only use haptic and kinematic observations from the robot's end effector, which are readily attainable. Collecting training data from real world physical human-robot interaction can be time consuming, costly, and put people at risk. Instead, we train our predictive model using data collected in an entirely self-supervised fashion from a physics-based simulation. We evaluated our approach with a PR2 robot that attempted to pull a hospital gown onto the arms of 10 human participants. With a 0.2s prediction horizon, our controller succeeded at high rates and lowered applied force while navigating the garment around a persons fist and elbow without getting caught. Shorter prediction horizons resulted in significantly reduced performance with the sleeve catching on the participants' fists and elbows, demonstrating the value of our model's predictions. These behaviors of mitigating catches emerged from our deep predictive model and the controller objective function, which primarily penalizes high forces.Comment: 8 pages, 12 figures, 1 table, 2018 IEEE International Conference on Robotics and Automation (ICRA

Modeling Humans at Rest with Applications to Robot Assistance

Humans spend a large part of their lives resting. Machine perception of this class of body poses would be beneficial to numerous applications, but it is complicated by line-of-sight occlusion from bedding. Pressure sensing mats are a promising alternative, but data is challenging to collect at scale. To overcome this, we use modern physics engines to simulate bodies resting on a soft bed with a pressure sensing mat. This method can efficiently generate data at scale for training deep neural networks. We present a deep model trained on this data that infers 3D human pose and body shape from a pressure image, and show that it transfers well to real world data. We also present a model that infers pose, shape and contact pressure from a depth image facing the person in bed, and it does so in the presence of blankets. This model similarly benefits from synthetic data, which is created by simulating blankets on the bodies in bed. We evaluate this model on real world data and compare it to an existing method that requires RGB, depth, thermal and pressure imagery in the input. Our model only requires an input depth image, yet it is 12% more accurate. Our methods are relevant to applications in healthcare, including patient acuity monitoring and pressure injury prevention. We demonstrate this work in the context of robotic caregiving assistance, by using it to control a robot to move to locations on a person’s body in bed.Ph.D

Multidimensional Capacitive Sensing for Robot-Assisted Dressing and Bathing

Robotic assistance presents an opportunity to benefit the lives of many people with physical disabilities, yet accurately sensing the human body and tracking human motion remain difficult for robots. We present a multidimensional capacitive sensing technique that estimates the local pose of a human limb in real time. A key benefit of this sensing method is that it can sense the limb through opaque materials, including fabrics and wet cloth. Our method uses a multielectrode capacitive sensor mounted to a robot's end effector. A neural network model estimates the position of the closest point on a person's limb and the orientation of the limb's central axis relative to the sensor's frame of reference. These pose estimates enable the robot to move its end effector with respect to the limb using feedback control. We demonstrate that a PR2 robot can use this approach with a custom six electrode capacitive sensor to assist with two activities of daily living-dressing and bathing. The robot pulled the sleeve of a hospital gown onto able-bodied participants' right arms, while tracking human motion. When assisting with bathing, the robot moved a soft wet washcloth to follow the contours of able-bodied participants' limbs, cleaning their surfaces. Overall, we found that multidimensional capacitive sensing presents a promising approach for robots to sense and track the human body during assistive tasks that require physical human-robot interaction.Comment: 8 pages, 16 figures, International Conference on Rehabilitation Robotics 201

Autobed: A Web-Controlled Robotic Bed

We (the Healthcare Robotics Lab at Georgia Tech) have developed an additional module for an Invacare fully electric hospital bed (Model 5410IVC) so that the bed can be controlled from a web-based interface. This module can be easily plugged between the hand control and the Invacare bed, without having to modify any existing hardware on the bed. We call a bed so modified an 'Autobed.' With this feature, users who are unable to operate the standard bed controls, but can access a web browser, are able to position the bed by themselves without having to rely on a caregiver (for example, patients with quadriplegia). This page describes how to make the Autobed module using relatively inexpensive, commercially available hardware. This document is a representation of the content provided at http://hsi.gatech.edu/hrl/project_autobed_v2.shtml as of February 15th, 2016, and is intended to create a lasting, citable, and archival copy of this material, which details the design and instructions for building the 'Autobed' device

3D Human Pose Estimation on a Configurable Bed from a Pressure Image

Robots have the potential to assist people in bed, such as in healthcare settings, yet bedding materials like sheets and blankets can make observation of the human body difficult for robots. A pressure-sensing mat on a bed can provide pressure images that are relatively insensitive to bedding materials. However, prior work on estimating human pose from pressure images has been restricted to 2D pose estimates and flat beds. In this work, we present two convolutional neural networks to estimate the 3D joint positions of a person in a configurable bed from a single pressure image. The first network directly outputs 3D joint positions, while the second outputs a kinematic model that includes estimated joint angles and limb lengths. We evaluated our networks on data from 17 human participants with two bed configurations: supine and seated. Our networks achieved a mean joint position error of 77 mm when tested with data from people outside the training set, outperforming several baselines. We also present a simple mechanical model that provides insight into ambiguity associated with limbs raised off of the pressure mat, and demonstrate that Monte Carlo dropout can be used to estimate pose confidence in these situations. Finally, we provide a demonstration in which a mobile manipulator uses our network's estimated kinematic model to reach a location on a person's body in spite of the person being seated in a bed and covered by a blanket.Comment: 8 pages, 10 figure

Efficient Algorithm on a Non-staggered Mesh for Simulating Rayleigh-Benard Convection in a Box

An efficient semi-implicit second-order-accurate finite-difference method is described for studying incompressible Rayleigh-Benard convection in a box, with sidewalls that are periodic, thermally insulated, or thermally conducting. Operator-splitting and a projection method reduce the algorithm at each time step to the solution of four Helmholtz equations and one Poisson equation, and these are are solved by fast direct methods. The method is numerically stable even though all field values are placed on a single non-staggered mesh commensurate with the boundaries. The efficiency and accuracy of the method are characterized for several representative convection problems.Comment: REVTeX, 30 pages, 5 figure

Order-of-magnitude speedup for steady states and traveling waves via Stokes preconditioning in Channelflow and Openpipeflow

Steady states and traveling waves play a fundamental role in understanding hydrodynamic problems. Even when unstable, these states provide the bifurcation-theoretic explanation for the origin of the observed states. In turbulent wall-bounded shear flows, these states have been hypothesized to be saddle points organizing the trajectories within a chaotic attractor. These states must be computed with Newton's method or one of its generalizations, since time-integration cannot converge to unstable equilibria. The bottleneck is the solution of linear systems involving the Jacobian of the Navier-Stokes or Boussinesq equations. Originally such computations were carried out by constructing and directly inverting the Jacobian, but this is unfeasible for the matrices arising from three-dimensional hydrodynamic configurations in large domains. A popular method is to seek states that are invariant under numerical time integration. Surprisingly, equilibria may also be found by seeking flows that are invariant under a single very large Backwards-Euler Forwards-Euler timestep. We show that this method, called Stokes preconditioning, is 10 to 50 times faster at computing steady states in plane Couette flow and traveling waves in pipe flow. Moreover, it can be carried out using Channelflow (by Gibson) and Openpipeflow (by Willis) without any changes to these popular spectral codes. We explain the convergence rate as a function of the integration period and Reynolds number by computing the full spectra of the operators corresponding to the Jacobians of both methods.Comment: in Computational Modelling of Bifurcations and Instabilities in Fluid Dynamics, ed. Alexander Gelfgat (Springer, 2018

Autobed: Open Hardware for Accessible Web-based Control of an Electric Bed

Individuals with severe motor impairments often have difficulty operating the standard controls of electric beds and so require a caregiver to adjust their position for utility, comfort, or to prevent pressure ulcers. Assistive human-computer interaction devices allow many such individuals to operate a computer and web browser. Here, we present the Autobed, a Wi-Fi-connected device that enables control of an Invacare Full-Electric Homecare Bed, a Medicare-approved device in the US, from any modern web browser, without modification of existing hardware. We detail the design and operation of the Autobed. We also examine its usage by one individual with severe motor impairments and his primary caregiver in their own home, including usage logs from a period of 102 days and detailed questionnaires. Finally, we make the entire system, including hardware design and components, software, and build instructions, available under permissive open-source licenses